Display device, projector, and display method

ABSTRACT

A display device including an image input unit that receives image data from an image supply device, a display unit that displays a display image on a display surface based on image data, a location detection unit that detects a pointed location with respect to the display image on the display surface, a coordinate calculation unit that calculates first coordinates as coordinates of the pointed location in a displayable area on the display surface, a drawing unit that draws an image based on the first coordinates calculated by the coordinate calculation unit, and an output unit that outputs the coordinates to the image supply device based on the first coordinates.

CROSS-REFERENCE

The present application is a continuation of U.S. patent applicationSer. No. 14/513,600 filed Oct. 14, 2014, which claims priority from U.S.patent application Ser. No. 13/604,727 filed Sep. 6, 2012 (now U.S. Pat.No. 8,878,801), which claims priority from Japanese Patent ApplicationNo. 2011-204289 filed Sep. 20, 2011, which are hereby incorporated byreference in their entirety.

BACKGROUND

1. Technical Field

The present invention relates to a display device that outputsinformation of a pointed location, a projector, and a display method.

2. Related Art

In related art, when a specific location of an image displayed by adisplay device such as a projector is pointed, a device of detecting thepointed location and displaying a pointer or the like in response to thedetected location has been known (for example, see Patent Document 1(Japanese Patent No. 4272904)). In this type of device, it is necessaryto perform calibration so that the pointed location and the displaylocation of the pointer or the like may coincide with each other.Generally, it is necessary to recalibrate at each time when the displaycondition changes in such a way that the display location of thedisplayed image changes, and lots of effort is required. Accordingly, inthe device disclosed in Patent Document 1, when the display locationchanges, location change data indicating a location relationship betweenbefore and after the change is used, and thereby, the calibration isomitted.

In the device disclosed in Patent Document 1, when the display conditionchanges, data indicating a relationship between before and after thechange is necessary. There are a wide variety of changes of displayconditions in the display device, and it is not easy to prepare dataindicating relationships between before and after the changes inadvance. Accordingly, in the device disclosed in Patent Document 1, ateach time when the display condition changes, data indicating arelationship between before and after the change is calculated, however,there has been a problem that the processing of calculating the data iscomplex and its load is greater, and thus, high arithmetic processingperformance is required.

SUMMARY

An advantage of some aspects of the invention is to provide a displaydevice, a projector, and a display method that can reduce an executionfrequency of calibration when a pointed location on a displayed image isdetected.

An aspect of the invention is directed to a display device including adisplay unit that displays a display image on a display surface based onimage data, a location detection unit that detects a pointed locationwith respect to the display image on the display surface, a coordinatecalculation unit that calculates first coordinates as coordinates of thepointed location in a displayable area within the display surface, acoordinate conversion unit that converts the first coordinatescalculated by the coordinate calculation unit into second coordinates ascoordinates in the image data, and an output unit that outputs thesecond coordinates obtained by the coordinate conversion unit.

According to this aspect of the invention, the coordinates of thepointed location with respect to the display image displayed on thedisplay surface based on the image data are output as the coordinates inthe image data, and thus, in equipment using the output coordinates, therelative position between the pointed location and the image data may bespecified without being affected by the display mode of displayresolution, the largeness of the display area, or the like. In theprocess of obtaining the coordinates of the pointed location in theimage data, it is not necessary to directly associate the image dataitself with the pointed location, and it is not necessary to performcalibration even when the size of the image data or the like is changed.Therefore, the execution frequency of the calibration may be reduced.

Another aspect of the invention is directed to the display devicedescribed above, wherein the coordinate conversion unit converts thefirst coordinates calculated by the coordinate calculation unit into thesecond coordinates based on image location information as informationindicating a location of the display image with respect to thedisplayable area.

According to this aspect of the invention, even when the image locationinformation as the information indicating the location of the displayimage with respect to the displayable area changes, the coordinates ofthe pointed location may be correctly converted and output.

Still another aspect of the invention is directed to the display devicedescribed above, wherein the coordinate conversion unit converts thefirst coordinates calculated by the coordinate calculation unit into thesecond coordinates based on resolution of the image data.

According to this aspect of the invention, even when the resolution ofthe image data changes, the coordinates of the pointed location may becorrectly converted and output.

Yet another aspect of the invention is directed to the display devicedescribed above, wherein the location detection unit detects a locationof a pointing tool on the display surface using an imaging unit, andthereby, detects the pointed location in the displayable area.

According to this aspect of the invention, the pointed location may bepromptly detected based on a taken image obtained by imaging of thedisplay surface.

Still yet another aspect of the invention is directed to the displaydevice described above, wherein the coordinate conversion unit usescoordinates of a location near the pointed location in the image data asconverted coordinates when the pointed location is not contained in thedisplay image.

According to this aspect of the invention, even when the location whereno image is displayed is pointed, the coordinates of the pointedlocation may be output. Further, the output coordinates are thecoordinates of the location near the pointed location, and may beprocessed in the same manner as that for the coordinates of the pointedlocation.

Further another aspect of the invention is directed to the displaydevice described above, wherein the coordinate conversion unit does notoutput converted coordinates when the pointed location is not containedin the display image.

According to this aspect of the invention, the coordinates correspondingto the pointed location are output only when an area with the displayimage is pointed, and thus, an operation in response to only thepointing on the location overlapping with the image may be performed.

Still further another aspect of the invention is directed to the displaydevice described above, wherein, when image processing is executed onthe image data and a display mode of the display image on the displaysurface is changed, the coordinate conversion unit converts the firstcoordinates into the second coordinates based on the image locationinformation changed in response to the display mode.

According to this aspect of the invention, proper coordinates maybeconstantly output with the change in display mode of the display image.

Yet further another aspect of the invention is directed to a projectorincluding a light modulator that modulates light generated by a lightsource, an image forming unit that forms a display image in a drawingarea of the light modulator based on image data, a projection unit thatprojects the display image formed by the image forming unit on aprojection surface, a location detection unit that detects a pointedlocation with respect to the display image on the projection surface, acoordinate calculation unit that calculates first coordinates ascoordinates of the pointed location in a projection area on theprojection surface, a coordinate conversion unit that converts the firstcoordinates calculated by the coordinate calculation unit into secondcoordinates as coordinates in the image data, and an output unit thatoutputs the second coordinates obtained by the coordinate conversionunit.

According to the projector of the aspect of the invention, thecoordinates of the pointed location with respect to the display imageprojected on the projection surface based on the image data are outputas the coordinates in the image data, and thus, in equipment using theoutput coordinates, the relative position between the pointed locationand the image data may be specified without being affected by thedisplay mode of display resolution, the largeness of the display area,or the like. In the process of obtaining the coordinates of the pointedlocation in the image data, it is not necessary to directly associatethe image data itself with the pointed location, and it is not necessaryto perform calibration even when the size of the image data or the likeis changed. Therefore, the execution frequency of the calibration may bereduced.

Still yet further another aspect of the invention is directed to adisplay method including displaying a display image on a display surfacebased on image data, detecting a pointed location with respect to thedisplayed display image, calculating first coordinates as coordinates ofthe pointed location in a displayable area within the display surface,converting the calculated first coordinates into second coordinates ascoordinates in the image data, and outputting the second coordinatesobtained by the conversion.

According to this aspect of the invention, the coordinates of thepointed location with respect to the display image displayed on thedisplay surface based on the image data are output as the coordinates inthe image data, and thus, in equipment using the output coordinates, therelative position between the pointed location and the image data may bespecified without being affected by the display mode of displayresolution, the largeness of the display area, or the like. In theprocess of obtaining the coordinates of the pointed location in theimage data, it is not necessary to directly associate the image dataitself with the pointed location, and it is not necessary to performcalibration even when the size of the image data or the like is changed.Therefore, the execution frequency of the calibration may be reduced.

A further another aspect of the invention is directed to a program thatcan be executed by a computer that controls a display device ofdisplaying a display image on a display surface based on image data, andallows the computer to function as a location detection unit thatdetects a pointed location with respect to the display image on thedisplay surface, a coordinate calculation unit that calculates firstcoordinates as coordinates of the pointed location in a displayable areawithin the display surface, a coordinate conversion unit that convertsthe first coordinates calculated by the coordinate calculation unit intosecond coordinates as coordinates in the image data, and an output unitthat outputs the second coordinates obtained by the coordinateconversion unit.

The program may be implemented as a recording medium in which theprogram is recorded in a computer-readable form.

According to the aspects of the invention, the coordinates of thepointed location with respect to the display image displayed on thedisplay surface may be output as the coordinates in the image datawithout direct association of the image data itself with the pointedlocation, and the execution frequency of the calibration may be reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 shows a configuration of a display system according to anembodiment of the invention.

FIG. 2 is a block diagram showing a functional configuration of aprojector.

FIG. 3 is a block diagram showing a functional configuration of a PC.

FIGS. 4A and 4B show examples of projection of images on a screen, andFIG. 4A shows a state in which a pointer is projected according to apointed location and FIG. 4B shows an example in which drawing isperformed according to the pointed location.

FIGS. 5A and 5B are explanatory diagrams showing processing of detectingand converting coordinates.

FIGS. 6A and 6B are explanatory diagrams showing the processing ofdetecting and converting the coordinates.

FIGS. 7A to 7C are explanatory diagrams showing changes of a projectionstate of an image and processing of converting coordinates.

FIGS. 8A to 8D are explanatory diagrams showing changes of a projectionstate of an image and processing of converting coordinates.

FIG. 9 is a flowchart showing an operation of the projector.

FIG. 10 is a block diagram showing a functional configuration of aprojector as a modified example.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

As below, embodiments to which the invention is applied will beexplained with reference to the drawings.

FIG. 1 shows a configuration of a display system 10 using a projector 11according to an embodiment.

The projector 11 as a display device is wired-connected to a PC(Personal Computer) 13 as an image supply device by an image signalcable or the like. Image data is input from the PC 13 to the projector11, and the projector 11 projects a display image on a screen SC as aprojection surface (display surface) based on the input image data.Further, the projector 11 is connected to the PC 13 by a communicationcable or the like and transmits and receives control data etc. betweenthe PC 13 and itself. The projector 11 may perform projection if theimage data input from the PC 13 represents a still image or a movingimage. The screen SC is not limited to a flat plate fixed to a wallsurface, but the wall surface itself may be used as the screen SC. Here,a range in which images are projected on the projector 11 is referred toas an effective projection area 11B (displayable area).

In the display system 10, during image projection by the projector 11, auser may hold a pointing tool 12 in his or her hand and execute alocation pointing operation in the effective projection area 11B of thescreen SC. The pointing tool 12 is an operation device having a penshape or a rod shape, and used for pointing an arbitrary location on thescreen SC. The projector 11 has a function of detecting a tip endlocation of the pointing tool 12, as will be described later, andoutputs control data indicating coordinates of the detected pointedlocation to the PC 13.

FIG. 2 is a block diagram showing a functional configuration of theprojector 11.

The projector 11 is roughly divided and includes an image processingunit 110 that executes image processing for display based on image datainput from the PC 13, a projection unit 3 (display unit) that projectsan image on the screen SC according to the control of the imageprocessing unit 110, a location detection unit 150 that detects apointed location of the pointing tool 12 on the screen SC, a coordinateconversion unit 160 that converts coordinates of the pointed locationdetected by the location detection unit 150 into coordinates in theimage data, an output unit 101 (output unit) that outputs the convertedcoordinates converted by the coordinate conversion unit 160 to the PC13, and a control unit 103 that controls the respective units.

The control unit 103 includes a CPU, a nonvolatile memory, a RAM, etc.(not shown), and reads out a control program 105A stored in a memoryunit 105 connected to the control unit 103 and controls the respectiveunits of the projector 11. Further, by executing the control program105A stored in the memory unit 105, the control unit 103 functions as acalibration execution part 103A. The calibration execution part 103Aexecutes calibration, which will be described later, and obtains acorrespondence relationship (coordinate conversion parameter) betweenthe coordinates in taken image data and the coordinates in the area onthe screen SC to be calibrated. The memory unit 105 includes a magnetic,optical recording device or a semiconductor memory device, and storesdata of various programs, various set values, etc. including the controlprogram 105A.

An operation panel 41 and a remote receiver unit 45 are connected to thecontrol unit 103.

The operation panel 41 includes various switches and indicator lamps andis provided on an exterior housing (not shown) of the projector 11. Thecontrol unit 103 appropriately lights or blinks the indicator lamps ofthe operation panel 41 in response to the operation condition and theset condition of the projector 11. When the switch of the operationpanel 41 is operated, an operation signal in response to the operatedswitch is output to the control unit 103.

Further, the projector 11 receives an infrared signal transmitted from aremote (not shown) used by the user as an operator who operates theprojector 11 in response to a button operation by the remote receiverunit 45. The remote receiver unit 45 receives the infrared signalreceived from the remote using a light receiving element, and outputs anoperation signal in response to the signal to the control unit 103. Theoperation panel 41, the remote, etc. form an operation part for the userto input the operation for the projector 11. The operation signalindicating the operation for the projector 11 may be transmitted fromthe PC 13 to the projector 11 and the projector 11 may be controlledbased on the operation signal. In this case, the PC 13 also functions asthe operation part for the user to input the operation for the projector11.

The control unit 103 detects the operation by the user based on theoperation signal input from the operation panel 41 or the remotereceiver unit 45, and controls the projector 11 according to theoperation.

The projector 11 is roughly divided and includes an optical system thatforms an optical image and an image processing system that electricallyprocesses an image signal. The optical system is a projection unit 30(projection unit) including an illumination system 31, a light modulator32 (light modulation unit), and a projection system 33. The illuminationsystem 31 includes a light source of a xenon lamp, ultrahigh pressuremercury lamp, an LED (Light Emitting Diode), or the like. Further, theillumination system 31 may include a reflector and an auxiliaryreflector that guide light generated by the light source to the lightmodulator 32, and may include a group of lenses (not shown) forimproving the optical property of the projection light, a polarizer, aphotochromic element that reduces the amount of light generated by thelight source in a path reaching the light modulator 32, or the like.

The light modulator 32 receives the signal from the image processingsystem, which will be described later, and modulates the light from theillumination system 31. In the embodiment, the case where the lightmodulator 32 is formed using a transmissive liquid crystal display panelwill be explained as an example. In this configuration, the lightmodulator 32 includes three liquid crystal display panels correspondingto the three primary colors of RGB for color projection. The light fromthe illumination system 31 is separated into three color lights of RGBand the respective color lights enter the corresponding respectiveliquid crystal display panels. The color lights modulated through therespective liquid crystal display panels are combined by a combiningsystem including a cross dichroic prism or the like and output to theprojection system 33.

The projection system 33 includes a zoom lens that enlarges, reduces,and focuses the projected image, a zoom adjustment motor that adjuststhe degree of zooming, a focus adjustment motor that performs adjustmentof focus, etc.

The projection unit 3 includes a projection system drive part 121 thatdrives the respective motors of the projection system 33 according tothe control of a display control part 107, a light modulator drive part119 that drives the light modulator 32 for drawing based on the imagesignal output from the display control part 107, and a light sourcedrive part 117 that drives the light source of the illumination system31 according to the control of the control unit 103 in addition to theprojection unit 30.

On the other hand, the image processing system includes an imageprocessing unit 110 that processes image data according to the controlof the control unit 103 that controls the entire projector 11 in anintegrated manner. The image processing unit 110 includes an image inputpart 104 connected to the PC 13. The image input part 104 is aninterface for inputting image data and, for example, general-purposeinterfaces such as a DVI (Digital Visual Interface) interface, a USBinterface, and a LAN interface to which digital video signals are input,an S-video terminal to which composite video signals such as NTSC, PAL,SECAM are input, an RCA terminal to which composite video signals areinput, a D-terminal to which component video signals are input, an HDMIconnector compliant to the HDMI (registered trademark) standard, etc.may be used. Further, the image input part 104 may have an A/D convertercircuit that converts analog video signals into digital image signalsand is connected to the PC 13 via an analog video terminal such as a VGAterminal. Note that the image input part 104 may transmit and receiveimage signals via wired communication or transmit and receive imagesignals via wireless communication.

Further, the image input part 104 may have a DisplayPort designed byVESA (Video Electronics Standards Association), and specifically have aDisplayPort connector or a Mini Displayport connector and an interfacecircuit compliant to the Displayport standard. In this case, theprojector 11 may be connected to the DisplayPort of the PC 13 or aportable device having the same function as that of the PC 13.

Furthermore, the image processing unit 110 (image forming unit) includesthe display control part 107 that processes the image data input via theimage input part 104, and an image processing part 113 that develops animage in a frame memory 115 according to the control of the displaycontrol part 107 and generates an image to be projected by theprojection unit 30.

The display control part 107 (a display control unit) performsdiscrimination of the format (frame rate, resolution, compression state)of the image data input via the image input part 104 or the like,determines necessary processing for displaying a display image on thelight modulator 32, and executes the processing by controlling the imageprocessing part 113. The image processing part 113 develops the imagedata input via the image input part 104 in the frame memory 115according to the control of the display control part 107, appropriatelyexecutes various conversion processing such as interlace/progressiveconversion and resolution conversion, generates an image signal in apredetermined format for displaying the display image drawn in the framememory 115, and outputs the signal to the display control part 107. Notethat the projector 1 may change the resolution and the aspect ratio ofthe input image data and display the data, or display the image data dotby dot with the resolution and the aspect ratio of the input image datamaintained. Further, the image processing part 113 may execute variousimage processing such as keystone correction, color compensation inresponse to the color mode, and enlarging/reducing processing accordingto the control of the display control part 107. The display control part107 outputs the image signal processed by the image processing part 113to the light modulator drive part 119, and displays the signal on thelight modulator 32. Further, the image processing part 113 derives imagelocation information, which will be described later, from information ofthe resolution, the aspect ratio of the image data being displayed, thedisplay size in the liquid crystal display panel of the light modulator32, etc., and outputs the obtained image location information to thecoordinate conversion unit 160.

The control unit 103 executes the control program 105A and controls thedisplay control part 107 to execute the keystone correction of thedisplay image formed on the screen SC. Further, the control unit 103controls the display control part 107 to execute the enlarging/reducingprocessing of the display image based on the operation signal input fromthe operation panel 41 or the remote receiver unit 45.

The projector 11 has the location detection unit 150 that detectscoordinates of the pointed location pointed by the pointing tool 12 onthe screen SC. The location detection unit 150 includes a locationdetection part 151 (location detection unit) having an imaging part 153that images the screen SC, an image control part 155 that controls theimaging part 153, and a location detection processing part 157 thatdetects the pointed location of the pointing tool 12 based on the takenimage of the imaging part 153, and a coordinate calculation part 159(coordinate calculation unit) that calculates the coordinates of thepointed location detected by the location detection part 151.

The imaging part 153 is a digital camera of taking an angle of viewincluding the maximum range in which the projection unit 30 can projectimages on the screen SC (corresponding to a projectable area 11A, whichwill be described later), and executes imaging according to the controlof the image control part 155 and outputs taken image data. The imagecontrol part 155 controls the imaging part 153 to execute imagingaccording to the control of the control unit 103. When the imaging part153 has mechanisms of adjusting zoom factor, focus, aperture at imaging,the image control part 155 controls these mechanisms to execute imagingunder preset conditions. After imaging, the image control part 155acquires the taken image data output by the imaging part 153 and outputsthe data to the location detection processing part 157. The taken imagedata output from the imaging part 153 may be expressed in the format ofRGB, YUV, or the like, or may indicate only the brightness component.Further, the image control part 155 may output the taken image dataoutput from the imaging part 153 to the location detection processingpart 157 without change, or adjust resolution or convert the data into apredetermined file format (JPEG, BMP, or the like) and output the datato the location detection processing part 157.

Note that the imaging part 153 may have a configuration that can imagevisible light or a configuration that can image non-visible light(infrared light or the like). In the case where the imaging part 153 canimage non-visible light, a configuration in which the pointing tool 12outputs non-visible light and the imaging part 153 images thenon-visible light output from the pointing tool 12 or a configuration inwhich the pointing tool 12 has a reflection part that can reflectnon-visible light, non-visible light is projected from the projector 11to the screen SC under the control of the control unit 103, and thenon-visible light reflected by the reflection part of the pointing tool12 is imaged by the imaging part 153, or the like may be employed.Further, a pattern for location detection may be provided on the surfaceof the pointing tool 12. In this case, by detecting the pattern forlocation detection from the taken image taken by the imaging part 153,the pointing tool 12 may be detected.

The location detection processing part 157 analyzes the taken image datainput from the image control part 155, and extracts a boundary betweenthe outside of the effective projection area 11B and the effectiveprojection area 11B and the image of the pointing tool 12 from the takenimage data and specifies the pointed location by the pointing tool 12.The pointed location of the pointing tool 12 is a location of the tipend of the rod-shaped or pen-shaped pointing tool 12, for example. Thelocation detection part 151 obtains coordinates of the detected pointedlocation in the effective projection area 11B.

Further, the projector 11 includes the coordinate conversion unit 160(coordinate conversion unit) that converts the coordinates output by thelocation detection unit 150 (first coordinates) into the coordinates inthe image data input from the PC 13 (second coordinates).

The coordinates output by the location detection processing part 157 arecoordinates detected based on the taken image data of the imaging part153, and coordinates on coordinate axes virtually provided on thedisplay image formed on the screen SC. The coordinate conversion unit160 acquires various information including resolution of the imagedeveloped by the image processing part 113 in the frame memory 115 andinformation on processing of resolution conversion, zooming, or the likeperformed when the image processing part 113 developed the image, andconverts the coordinates on the display image obtained by the locationdetection processing part 157 into coordinates in the input image databased on the acquired information. As described above, the lightmodulator 32 is formed using the liquid crystal display panel having apredetermined number of pixels arranged laterally and longitudinally ina matrix, for example, and, by placing the coordinate axes of thevirtual orthogonal coordinate system in the arrangement directions ofthe pixels, the location on the panel may be expressed by thecoordinates. On the other hand, the coordinates in the taken image dataare affected by various elements such as the distance between theprojector 11 and the screen SC, the zoom factor in the projection system33, the installation angle of the projector 11, and the distance betweenan imaging device 5 and the screen SC, etc. Accordingly, in theprojector 11 according to the embodiment of the invention, calibration,which will be described later, is first executed, and a correspondencerelationship (coordinate conversion parameter) between the coordinatesin the taken image data and the coordinates in the area on the screen SCto be calibrated is obtained. Here, the area on the screen SC to becalibrated may be the entire effective projection area 11B or a part ofthe effective projection area 11B. As the case where the part of theeffective projection area 11B is calibrated, the case where, when theaspect ratio of the display image of the projector 11 and the aspectratio of the screen SC are different (for example, the displayresolution of the projector 11 is WXGA and the aspect ratio of thescreen SC is 4 : 3), display is performed so that the width in thevertical direction of the display image of the projector 11 may be thesame as the width in the vertical direction of the screen SC isconsidered. In this case, it is conceivable that, of the effectiveprojection area 11B of the projector 11, the area contained in thescreen SC is to be calibrated and the other areas are not to becalibrated. When the coordinate conversion parameter is obtained by thecalibration execution part 103A, the coordinate calculation part 159performs conversion of the coordinates based on the coordinateconversion parameter. The conversion processing will be described later.Further, the coordinate conversion unit 160 converts the coordinatesoutput from the coordinate calculation part 159 (first coordinates)based on the image location information, which will be described later,and outputs the converted coordinates (second coordinates) to the outputunit 101.

The output unit 101 is an interface connected to the PC 13 andoutputting the coordinate data after conversion processing by thecoordinate conversion unit 160 (coordinate information) to the PC 13,and, for example, includes a general-purpose interface such as a USBinterface, a wired LAN interface, a wireless LAN interface, or IEEE1394. Here, the image input part 104 and the output unit 101 will beexplained as separate functional blocks, however, obviously, they may bephysically integrated into one interface. For example, one USB interfacemay realize both functions of the output unit 101 and the image inputpart 104. Further, the output unit 101 may be connected to the imageprocessing part 113 of the image processing unit 110 and may output thecoordinates after the conversion processing of the coordinate conversionunit 160 to the image processing unit 110. The output destination of theoutput unit 101 is controlled by the control unit 103. The coordinatedata output by the output unit 101 is output to the PC 13 as the samedata as coordinate data output by a pointing device such as a mouse, atrackball, a digitizer, a pen tablet, or the like.

Further, in the PC 13, in the case where the coordinate data output fromthe output unit 101 is treated equally to the coordinate data output bythe general-purpose pointing devices, general-purpose device driverprograms corresponding to these general-purpose pointing devices may beused. Generally, these general-purpose device driver programs areinstalled as part of the OS (operating system) of the PC 13 in advance,and thus, it is not necessary to install device driver programs when thegeneral-purpose device driver programs are used. Further, it is notnecessary to prepare specialized device driver programs because thegeneral-purpose device driver programs are used. On the other hand, theinformation that can be exchanged between the projector 11 and the PC 13is limited in the range defined by the specifications of thegeneral-purpose device driver programs.

Alternatively, specialized device driver programs compliant to theprojector 11 may be prepared and the device driver programs may beinstalled and used in the PC 13. In this case, the specialized devicedriver programs are necessary, but the information that can be exchangedbetween the projector 11 and the PC 13 may be arbitrarily set inresponse to the specifications of the specialized device driverprograms.

FIG. 3 is a block diagram showing a functional configuration of the PC13.

As shown in FIG. 3, the PC 13 includes a CPU 131 that executes thecontrol programs and centrally controls the respective parts of the PC13, a ROM 132 that stores a basic control program to be executed by theCPU 131 and data on the program, a RAM 133 that temporarily stores theprograms and the data executed by the CPU 131, a storage unit 134 thatstores the programs and the data in a non-volatile manner, an input unit135 that detects an input operation and outputs data and an operationsignal indicating input contents to the CPU 131, a display unit 136 thatoutputs display data for displaying processing results by the CPU 131etc., and an external I/F 137 that transmits and receives data etc.between an external device and itself, and these respective units areconnected to one another via a bus.

The input unit 135 includes an input I/F 141 having a connector and apower supply circuit, and an input device 142 is connected to the inputI/F 141. The input I/F 141 includes a general-purpose interface forinput device such as an USB interface, for example, and the input device142 is a keyboard or a pointing device such as a mouse or a digitizer.

A communication cable in connection to the projector 11 is connected tothe input I/F 141, and the coordinates of the pointed location by thepointing tool 12 are input from the projector 11. Here, to the input I/F141, the coordinate data output by the output unit 101 of the projector11 is input as the same data as the coordinate data output by thepointing device such as a mouse, a trackball, a digitizer, or a pentablet. Therefore, the PC 13 may process the coordinate data input fromthe projector 11 as an input signal from the input device, and mayperform an operation of moving the mouse cursor and the pointer based onthe coordinate data, for example.

The display unit 136 includes an image output I/F 143 having a connectorfor image signal output or the like, and image signal cables (not shown)in connection to a monitor 144 and the projector 11 are connected to theimage output I/F 143. The image output I/F 143 has pluralities of VGAterminals to which analog video signals are input, DVI interfaces towhich digital video signals are input, USB interfaces, LAN interfaces,S-video terminals to which composite video signals of NTSC, PAL, SECAM,etc. are input, RCA terminals to which composite video signals areinput, D-terminals to which component video signals are input, HDMIconnectors compliant to the HDMI (registered trademark) standard, etc.,for example, and the monitor 144 and the projector 11 are respectivelyconnected to the connectors. Further, the image output I/F 143 may havea DisplayPort designed by VESA, and specifically have a DisplayPortconnector or a Mini Displayport connector and an interface circuitcompliant to the Displayport standard. In this case, the PC 13 mayoutput digital video signals to the projector 11, the monitor 144, oranother device via the Displayport. Note that the image output I/F 143may transmit and receive image signals via wired communication ortransmit and receive image signals via wireless communication.

The storage unit 134 stores a display control program 13A to be executedby the CPU 131, and image data 13B to be output at execution of thedisplay control program 13A. The CPU 131 executes the display controlprogram 13A, and then, executes processing of transmitting the imagedata 13B to the projector 11. In this processing, the CPU 131 reproducesthe image data 13B, and generates an image signal with predetermineddisplay resolution using the display unit 136 and outputs the signal tothe image output I/F 143. Here, the display unit 136 outputs an analogimage signal to the connector to which an analog signal is output andoutputs digital image data to the connector to which digital image datais output.

Further, when the coordinates in response to the operation of thepointing device are input from the input unit 135 during the executionof the display control program 13A, the CPU 131 generates an image fordisplaying a pointer 12A (FIG. 1) in a location corresponding to thecoordinates. Then, the CPU 131 generates image data with the pointer 12Asuperimposed on the image data 13B being reproduced, and outputs theimage data from the input I/F 141 to the projector 11.

As described above, in the display system 10, the function of the PC 13drawing the image data with the pointer 12A superimposed thereon to beoutput to the projector 11 is executed by the PC 13.

FIGS. 4A and 4B show examples of projection of images on the screen SC,and FIG. 4A shows a state in which the pointer 12A is projectedaccording to the pointed location of the pointing tool 12 and FIG. 4Bshows a state in which a drawn FIG. 12C is drawn according to thepointed location.

When a display image is projected using the entire liquid crystaldisplay panels of the light modulator 32, an image is formed in theprojectable area 11A shown by a dashed-two dotted line in FIG. 4A.Trapezoidal distortion is produced as shown in FIG. 4A except the casewhere the projector 11 is positioned right in front of the screen SC,and the projector 11 performs keystone correction using the function ofthe display control part 107. After the execution of the keystonecorrection, a display image is projected in the effective projectionarea 11B. Typically, the effective projection area 11B is set to form arectangular shape with the maximum size on the screen SC. Specifically,the size is determined by the resolution of the liquid crystal displaypanels of the light modulator 32 and the degree of the trapezoidaldistortion and not necessarily the maximum size.

The calibration execution part 103A of the projector 11 executescalibration in the effective projection area 11B after the keystonecorrection. In the calibration, the calibration execution part 103Acontrols the image processing part 113 to draw a predetermined image forcalibration. In the state in which the image for calibration isprojected on the screen SC, the location detection unit 150 images thescreen SC under the control of the calibration execution part 103A. Theimage for calibration is an image in which dots are arranged on a whitebackground, for example, and stored in the memory unit 105 or the likein advance. Note that the image for calibration is not necessarilystored in the memory unit 105 or the like, but the calibration executionpart 103A may generate an image for calibration at each time whenexecution of calibration is necessary and the calibration is executed.

The calibration execution part 103A detects a contour of the displayimage in the taken image data, i.e., a boundary between the outside ofthe effective projection area 11B and the effective projection area 11Band dots in the taken image data, and specifies a correspondencerelationship among an imaging range (angle of view) of the locationdetection unit 150, i.e., a location in the taken image data, a locationon the effective projection area 11B, and a location on the image drawnby the image processing part 113. The calibration execution part 103Aobtains a coordinate conversion parameter used by the coordinatecalculation part 159 as will be described later based on thecorrespondence relationship between the location on the taken imagespecified by the calibration and the location on the effectiveprojection area 11B. The coordinate conversion parameter includes dataassociating coordinates on the image drawn by the image processing part113 with coordinates obtained on the taken image data. The coordinatecalculation part 159 may convert the coordinates obtained on the takenimage data into the coordinates on the image drawn by the imageprocessing part 113 based on the coordinate conversion parameter. Thecoordinate calculation processing is performed based on the coordinateconversion parameter.

The calibration is performed by execution of a program for calibration(not shown) stored in the memory unit 105 by the control unit 103, andthus, it is not necessary to install and execute the program forcalibration in the PC 13. Further, the calibration may be processingautomatically performed by the calibration execution part 103A based onthe taken image data or processing requiring use's operation for theimage for calibration. Furthermore, the projector 11 may use the twokinds of processing in combination. As a conceivable operation for theimage for calibration by the user, there is an operation of pointing adot contained in the image for calibration by the user using thepointing tool 12 or the like.

The location detection unit 150 of the projector 11 executes imaging inthe state in which the image is projected in the effective projectionarea 11B, virtually sets orthogonal coordinates with their origin at acorner of the effective projection area 11B in the taken image as shownby dashed arrows in the drawings, and obtains coordinates of the tip endlocation of the pointing tool 12 in the coordinate system. Theorthogonal coordinates are set based on the coordinate conversionparameter obtained by the calibration. Subsequently, when thecoordinates of the tip end of the pointing tool 12 in the image datadisplayed in the effective projection area 11B are obtained by thecoordinate conversion unit 160, the pointer 12A and a menu bar 12B shownin FIG. 4A, for example, are displayed according to the coordinates. Thepointer 12A is drawn as a sign indicating the tip end location of thepointing tool 12. Further, the menu bar 12B is a GUI that can beoperated by the pointing tool 12, and drawing of a figure such as aline, saving, erasing, and copying of data of the drawn figure, or thelike may be performed by pointing a button located on the menu bar 12Busing the pointing tool 12. As a specific example, by moving thepointing tool 12 from the location shown in FIG. 4A to the location inFIG. 4B, a drawn FIG. 12C is drawn along a trace of the tip end of thepointing tool 12. The drawn FIG. 12C is drawn by the PC 13 according tothe coordinate data indicating the pointed location of the pointing tool12 like the pointer 12A and the menu bar 12B, for example.

Further, on the menu bar 12B, a button for control of slide show displayof sequentially displaying plural images that can be externally supplied(for example, image data stored by an external storage device such as aUSB flash memory connected to the USB interface of the external I/F 102or the like), a button for settings on the function itself of theprojector 11 (changes of the aspect ratio, changes of the color mode,etc.) or the like can be placed. When the pointed location of thepointing tool 12 is output from the coordinate conversion unit 160, thecontrol unit 103 acquires its coordinates, specifies the button pointedin the menu bar 12B, and executes an operation in response to thepointing operation.

FIGS. 5A and 5B and FIGS. 6A and 6B are explanatory diagrams showingprocessing of detecting coordinates of a pointed location and convertingthem into coordinates in image data by the projector 11. FIG. 5A showsan initial state of the series of operation, FIG. 5B and FIGS. 6A and 6Bshow states in which resolution of the display image has been changedfrom the state in FIG. 5A.

In the example shown in FIG. 5A, the resolution of the effectiveprojection area 11B is 1280×800 dots and the resolution of the imagedata input from the PC 13 is also 1280×800 dots based on the resolutionof the liquid crystal display panels of the light modulator 32.Therefore, in the effective projection area 11B, a display image 201 of1280×800 dots is displayed. The location detection unit 150 sets an X-Yorthogonal coordinate system with the origin at the upper left corner ofthe effective projection area 11B, the rightward direction in the X-axisdirection, and the downward direction in the Y-axis direction, and setsthe end location in the X direction of the display image 201 to Xlmax,the end location in the Y direction to Ylmax, and coordinates of thepointed location of the pointing tool 12 to (X1 n, Y1 n).

When the image data input from the PC 13 is switched to a display image202 with resolution of 1024×768 dots, the display image 202 of 1066×800dots is projected on the screen SC as shown in FIG. 5B. The displayimage 202 has the lower resolution than that of the display image 201,and thus, the area in which the display image 202 is projected issmaller than the effective projection area 11B.

Here, as shown in FIGS. 5A and 5B, when the pointing tool 12 on thescreen SC is not moved, the pointed location itself is not moved, butthe relative position between the pointed location and the displayedimage changes. Accordingly, when the location detection unit 150displays the pointer 12A according to the coordinates (X1 n, Y1 n) ofthe pointed location in the effective projection area 11B calculatedbased on the taken image data of the imaging part 153, the pointer 12Ashifts from the actual pointed location.

That is, as shown in FIG. 6A, when the pointer is displayed at thecoordinates (X1 n, Y1 n) in the coordinate system with the origin at theupper left corner of the changed display image 202, a pointer 12A′ apartfrom the tip end of the pointing tool 12 is displayed. In this manner,the coordinates obtained with reference to the effective projection area11B are affected by the resolution of the display image, and it may beimpossible for the PC 13 to use the coordinates calculated by thelocation detection unit 150 for the display of the pointer 12A.

Accordingly, in order to deal with the case where the resolution of thedisplay image changes, the projector 11 performs processing ofconverting coordinates (X1 n, Y1 n) of the pointed location calculatedby the coordinate calculation part 159 of the location detection unit150 into coordinates (X2 n, Y2 n) of the pointed location in the displayimage being displayed using the coordinate conversion unit 160.

As below, specific processing will be explained.

In the embodiment, the coordinate conversion unit 160 expresses thecoordinates in the display image in a coordinate system (FIG. 5A) withthe origin set at the corner of the effective projection area 11B. Asshown in FIGS. 5B, 6A, and 6B, in the case where the display image(here, the display image 202) is displayed in an area smaller than theeffective projection area 11B, the location detection processing part157 detects the pointed location with the origin at the corner of thedisplay image in the taken image of the imaging part 153, and thecoordinate calculation part 159 specifies the location of the displayimage 202 in the effective projection area 11B and calculates thecoordinates (X1 n, Y1 n) in the effective projection area 11B.

The coordinate conversion unit 160 acquires image location informationfrom the image processing part 113 and obtains coordinates (X1 bmin, Y1bmin) of the upper left corner corresponding to the origin of thedisplay image 202 after change. The coordinates (X1 bmin, Y1 bmin) arecoordinates with the origin at the upper left corner of the effectiveprojection area 11B.

Further, in the following computation, values of X2 max, X2 min areused. The X2 max is the maximum value in the X-axis direction in thecoordinate system with the origin at the upper left corner of thedisplay image 202 when the display image 202 is displayed, and the X2min is the minimum value in the same coordinate system. That is, it isconsidered that the X2 max is the coordinate at the right end of thedisplay image 202 on the X-axis and the X2 min is the origin, zero,however, normalized values are used as the values of X2 max, X2 min andnot limited to X2 min=0. Accordingly, computation is performed with thevalue as a variable X2 min.

As shown in FIG. 6B, it is assumed that the coordinates of the upperleft corner corresponding to the origin of the display image 202 are (X1bmin, Y1 bmin), the coordinate value of the end in the X-axis directionof the effective projection area 11B is X1 max, the end in the X-axisdirection of the display image 202 is X1 bmax, the coordinate value ofthe end in the Y-axis direction of the effective projection area 11B isY1 max, and the end in the Y-axis direction of the display image 202 isY1 bmax.

In this case, coordinates (X2 n, Y2 n) are calculated by the followingequations (1), (2).

X2n=(X2max−X2min)×(X1n−X1bmin)÷(X1bmax−X1bmin)   (1)

Y2n=(Y2max−Y2min)×(Y1n−Y1bmin)÷(Y1bmax−Y1bmin)   (2)

In the embodiment, as shown in FIG. 6B, Y1 bmin=Y2 min=0 and Y1 bmax=Y2max=Y1 max. Accordingly, from the equation, Y2 n=Y1 n.

Actually, the coordinates of the pointed location are obtained asnormalized logical coordinates. As an example, X1 min=0, X1 max=32767,Y1 min=0, and Y1 max=32767.

Further, in the following example, the effective projection area 11B isset according to the image with resolution of 1280×800 dots and, whenthe coordinates in the effective projection area 11B are expressed by(XPn, YPn), it is assumed that (XPmin≦XPn≦XPmax, YPmin≦YPn≦YPmax) holdsand XPmin=0, XPmax=1280, YPmin=0, and YPmax=800.

Furthermore, as information on the location and the size of the displayimage displayed in the effective projection area 11B, it is assumed thatcoordinates at the upper right end of the display image is (XP0, YP0)and (XP0, YP0)=(0, 0) in this example, and the size in the X-axisdirection of the display image is WP0=1280 and the size in the Y-axisdirection of the display image is HP0=800.

The coordinates (X1 bmin, Y1 bmin) of the upper left corner and thecoordinates of the end location (X1 bmax, Y1 bmax) of the display imagein the effective projection area 11B are obtained by the followingequations (3) to (6).

X1bmin=(X1max−X1min)×XP0÷(XPmax−XPmin)   (3)

X1bmax=(X1max−X1min)×(XP0+WP0)÷(XPmax−XPmin)   (4)

Y1bmin=(Y1max−Y1min)×YP0÷(YPmax−YPmin)   (5)

Y1bmax=(Y1max−Y1min)×(YP0+HP0)÷(YPmax−YPmin)   (6)

The computations of the above equations (1) and (2) are performed basedon the values obtained by the equations (3) to (6), and the coordinateconversion unit 160 obtains the coordinates of the pointed location inthe display image. The coordinates may be used as information forspecifying the location in the image data when the PC 13 draws thepointer 12A, the menu bar 12B, or the drawn FIG. 12C in the image datato be processed. Accordingly, the pointer 12A, the menu bar 12B, and thedrawn FIG. 12C may be correctly drawn according to the pointed locationby the pointing tool 12 without being affected by the resolution, thezooming factor, or the like of the display image.

However, the location and the size of the display image displayed in theeffective projection area 11B are affected by the resolution and thedisplay location of the display image. For example, when the projector11 executes processing of changing the projection state such as changingof the display resolution, changing of the aspect ratio, zooming,changing (moving) of the display location of the image, or multi-windowdisplay processing in response to the operation using the operationpanel 41 or the remote receiver unit 45 or the control signaltransmitted from the PC 13, the image location information (XP0, YP0,WP0, HP0) also changes. Here, the image location information isinformation on arrangement of image arrangement areas (areas in whichthe display images 201, 202 are projected (displayed)) with respect tothe effective projection area 11B. In other words, the image locationinformation is information indicating location (arrangement) of thedisplay images with respect to the effective projection area 11B(displayable area). Further, when the display resolution of the PC 13changes and the resolution of the image data output to the projector 11by the PC 13 (for example, when the setting on the resolution is changedin the PC 13), the image location information also changes. Note that“multi-window display processing” refers to processing of dividing theeffective projection area 11B of the projector 11 into plural areas anddisplaying different images input from plural image supply devices inthese areas.

FIGS. 7A to 7C and FIGS. 8A to 8D are explanatory diagrams showingchanges of the projection state of an image and processing of convertingcoordinates, and showing examples in which the image locationinformation (XP0, YP0, WP0, HP0) changes due to changes of theprojection state.

In FIG. 7A, the display image 201 with the same resolution (1280×800) asthat of the effective projection area 11B is displayed. The imagelocation information in this case is (XP0=0, YP0=0, WP0=1280, HP0=800).Here, when the display image is changed to the display image 202 withthe different resolution (1066×800), as shown in FIG. 7B, non-displayareas 11C are produced around the display image 202. In this case, theimage location information is (XP0=107, YP0=0, WP0=1066, HP0=800).

Here, when the aspect ratio of the display image 202 is changed forenlarged display in the entire effective projection area 11B, as shownin FIG. 7C, the display image 202 is displayed to fill the effectiveprojection area 11B, and the image location information is (XP0=0,YP0=0, WP0=1280, HP0=800).

In the case where the non-display area 11C is produced and the pointedlocation of the pointing tool 12 is superimposed on the non-display area11C, the coordinate conversion unit 160 may not output the coordinatesof the pointed location or may output the coordinates of the locationnearest the pointed location within the range of the display image tothe PC 13.

Specifically, the coordinate conversion unit 160 determines whether ornot the coordinates calculated by the coordinate calculation part 159correspond to the non-display area 11C based on the image locationinformation before coordinate conversion processing. Here, when thecoordinates calculated by the coordinate calculation part 159 correspondto the non-display area 11C, the coordinate conversion unit 160determines whether or not the coordinates in the X-axis direction andthe coordinates in the Y-axis direction respectively correspond to thenon-display area 11C (whether or not the coordinates are contained inthe effective projection area 11B), and, when the coordinates correspondto the non-display area 11C, determines which of the larger coordinateside or the smaller coordinate side contains the coordinates in thenon-display area 11C. For example, in FIG. 7B, when the pointed locationis superimposed on the left non-display area 11C of the display image202, the coordinates in the X-axis direction of the pointed location iscontained in the non-display area 11C with the smaller values. When thecoordinate conversion unit 160 has determined a deviation direction withrespect to one of the coordinates in the X-axis direction and thecoordinates in the Y-axis direction, the unit assigns the coordinates ofthe end location of the display image 202 in the deviation direction tothe coordinates of the pointed location. When the pointed location issuperimposed on the left non-display area 11C of the display image 202in FIG. 7B, the value of the coordinate X1 n in the X-axis direction ofthe pointed location is changed to the value of the Xlbmin. Similarly,when the pointed location is superimposed on the right non-display area11C of the display image 202, the value of the coordinate X1 n in theX-axis direction of the pointed location is changed to the value of theXlbmax. The same may apply to the Y-axis direction.

That is, when the coordinates (X1 n, Y1 n) calculated by the coordinatecalculation part 159 do not satisfy (X1 bmin≦X1 n≦X1 bmax, Y1 bmin≦Y1n≦Y1 bmax), the coordinate conversion unit 160 outputs one of (X1 bmin,Y1 n), (X1 bmax, Y1 n), (X1 n, Y1 bmin), (X1 n, Y1 bmax). Thereby, withrespect to the pointed location not contained in the display image, thecoordinates may be output and the pointer 12A or the menu bar 12B may bedrawn near the pointed location.

Further, the display location of the display image 201 is shifted fromthe state shown in FIG. 7A to the left by the amount of 160 dots, asshown in FIG. 8A, the left side of the display image 201 is out of thescreen and the non-display area 11C is produced on the right of thedisplay image 201. In the state in FIG. 8A, the image locationinformation is (XP0=−160, YP0=0, WP0=1280, HP0=800). Note that, in FIG.8A, the case where the display location of the display image 201 isshifted to the left is exemplified, however, the display image 201 maybemoved in another direction than the left (right, upward, or downward).

In addition, the projector 11 has the so-called multi-window displayfunction of displaying plural display images side by side at the sametime. FIG. 8B shows an example of displaying the display image 202 and adisplay image 203 by the multi-window display function. In this example,the display image 202 and the display image 203 are reduced with theiraspect ratios kept so as to be displayed in the effective projectionarea 11B side by side, and the non-display area 11C is produced aroundthem. In the case where the plural display images are displayed at thesame time by the multi-window display function, the image locationinformation maybe defined on the respective display images. In the caseas shown in FIG. 8B, different image location information may be definedon the respective display image 202 and display image 203. Theresolution of the display image 201 after reduction becomes 533×400 inhalves in the longitudinal and lateral directions, and the imagelocation information on the display image 202 is (XP0=53, YP0=200,WP0=533, HP0=400).

The projector 11 can enlarge or reduce one of the respective displayimage 202 and the display image 203 at execution of the multi-windowdisplay function. In this case, when the user performs operation ofcommanding enlargement or reduction of one of the display images 202,203 with the pointing tool 12, the projector 11 enlarges or reduces thepointed display image in response to the operation, and updates theimage location information of the enlarged or reduced display image.

The projector 11 has a zoom function of enlarging an image to largerthan the effective projection area 11B and displaying a part thereof.FIG. 8C shows an example in which the display image 202 shown in FIG. 7Bis enlarged with resolution 1.25 times the original resolution. In theexample in FIG. 8C, a virtual display area 11D larger than the effectiveprojection area 11B is necessary for display of the entire display image202, and actually, only the part within the effective projection area11B at the center of the display image 202 is displayed. The imagelocation information is determined with reference to the coordinates ofthe corner of the virtual display area 11D and the resolution of thevirtual display area 11D to be (XP0=−27, YP0=−100, WP0=1333, HP0=1000).Further, the display location in the display image enlarged by the zoomfunction may be shifted. FIG. 8D shows a state in which the enlargeddisplay image 202 shown in FIG. 8C is shifted downward by an amount of100 dots. The processing corresponds to processing of moving the virtualdisplay area 11D downward relative to the effective projection area 11B,and the image location information is (XP0=−27, YP0=0, WP0=1333,HP0=1000). Note that, in FIG. 8A, the case where the display location ofthe display image 201 is shifted downward is exemplified, however, thedisplay image 201 may be moved in another direction than the downward(upward, right, or left).

The coordinate conversion unit 160 acquires information from the controlunit 103 and the display control part 107, updates the image locationinformation, and converts the coordinates based on the updated imagelocation information at each time when the projection state (displaystate) of the display image by the projection unit 30 changes. Forexample, the image location information is updated at the followingtimes

when the control unit 103 detects input of image data from the PC 13

when the control unit 103 detects a change in information on image datainput from the PC 13 (resolution of an image or the like)

when the resolution of image data is changed in the projector 1

when the aspect ratio of image data is changed

when a digital zoom function of enlarging/reducing an image drawn by thelight modulator 32 by image processing of image data to be projected isexecuted or terminated

when the display location of a display image with respect to theeffective projection area 11B is changed

when an image is enlarged by the digital zoom function, and a functionof changing the display location of the image by image processing isexecuted or terminated

when a tele/wide function of enlarging/reducing the projection size ofthe whole including the images drawn by the light modulator 32 and thebackground, i.e., the entire effective projection area 11B by performingimage processing of image data is executed or terminated

when an image is reduced by the digital zoom function, and a pictureshift function of changing the display location of the image by imageprocessing is executed or terminated

when simultaneous display of plural images is executed or terminated

when an output destination to which coordinates are output from thecoordinate conversion unit 160 is changed from the image processing unit110 to the PC 13 (output unit 101) or vise versa

All of changing of the resolution, changing of the aspect ratio, andexecution and termination of the various functions are executed by theimage processing unit 110 under the control of the control unit 103.Note that the listed times are just examples and, obviously, imagelocation information can be updated at other times.

FIG. 9 is a flowchart showing an operation of the projector 11, andspecifically shows an operation of detecting a pointed location by thepointing tool 12 and outputting coordinates of the pointed location.

The operation shown in FIG. 9 is repeatedly executed at regular timeintervals after the projector 11 is activated or when display of thepointer 12A and the menu bar 12B is commanded by an operation of theoperation panel 41 or the remote receiver unit 45.

First, whether or not calibration is necessary is determined (step S11).The determination may be performed according to the user's commandindicating whether or not calibration is necessary. Or, whether or notcalibration is necessary may be automatically determined by thecalibration execution part 103A and the calibration may be automaticallyperformed based on the determination result. If calibration is necessary(step S11; Yes), the calibration is executed as has been explained withreference to FIG. 4A (step S12). That is, an image for calibration isdrawn by the image processing part 113, imaging is executed by thelocation detection unit 150 with the image for calibration projected,the contour of the effective projection area 11B in the obtained takenimage data and feature points (dots or the like) contained in the imagefor calibration are detected, and thereby, the correspondencerelationship between the image drawn by the image processing part 113and the taken image data is obtained. Note that it is necessary toperform the calibration only once after the start of use of theprojector 11, and not necessary to perform it again unless a specificevent occurs. For example, in the cases of the following (1) to (3), itis necessary to perform new calibration.

(1) where keystone correction has been performed

(2) where an install condition of the projector 11 is changed, forexample, where the relative position (including the direction) of theprojector 11 with respect to the screen SC has been changed

(3) where an optical condition has been changed, for example, where thefocus or zoom condition of the projection system 33 has been changed andwhere the optical axis of the projection system 33 or the imaging part153 has been varied due to change with time or the like

If these events occur, the correspondence relationship between thelocation on the taken image data in the initial state and the locationon the image drawn by the image processing part 113 as reference forcalculation of coordinates by the coordinate conversion unit 160changes, and it is necessary to newly perform. calibration. If theseevents do not occur, it is not necessary to perform calibration again.If the events have not occurred after the previous use of the projector11 before the use at this time, the coordinate conversion parameterobtained in the previous calibration may be reused without newcalibration. Methods for the calibration execution part 103A todetermine whether or not calibration is necessary include, for example,a method of determining it based on whether or not there is an operationof the switch for commanding execution of keystone correction in theoperation panel 41, and a method of providing a sensor of detecting atilt or motion in the projector 11 and determining it based on a changein detection value of the sensor. Or, when adjustment of focus or zoomin the projection system 33 is performed, the calibration execution part103A may automatically execute the calibration. Or, for the user to knowa change in installation location and optical condition of the projector11 and perform the operation of commanding calibration execution, acorresponding switch may be provided on the operation panel 41 or theoperation part of the remote or the like.

When the image control part 155 allows the imaging part 153 to image therange containing the effective projection area 11B under the control ofthe control unit 103, the location detection processing part 157acquires the taken image data (step S13) and detects the pointedlocation of the pointing tool 12 based on the taken image data (stepS14) . Subsequently, the coordinate calculation part 159 calculates thecoordinates of the pointed location detected by the location detectionprocessing part 157 (step S15). The coordinates calculated at step S15are coordinates in the effective projection area 11B and the coordinates(Xln, Yin) explained in FIG. 5A.

The coordinate conversion unit 160 determines whether or not updating ofthe image location information is necessary (step S16) and, if updatingis necessary, acquires information from the control unit 103 and thedisplay control part 107 and updates the image location information(step S17). The processing at step S17 may be executed not limited atthe time after step S15, but at the above exemplified times as the needarises.

Then, the coordinate conversion unit 160 performs processing ofconverting the coordinates calculated by the coordinate calculation part159 into coordinates in the image data of the display image (step S18).The coordinates after conversion are the coordinates (X2n, Y2n)explained in FIG. 5B.

The coordinate conversion unit 160 outputs the converted coordinates tothe PC 13 (step S19), and the process is ended.

As described above, in the display system 10 according to the embodimentto which the invention is applied, the projector 11 includes theprojection unit 3 that displays the display image on the screen SC basedon the image data, the location detection part 151 that detects thepointed location with respect to the display image on the screen SC, thecoordinate calculation part 159 that calculates the first coordinates asthe coordinates of the pointed location in the displayable area withinthe screen SC (for example, the effective projection area 11B), thecoordinate conversion unit 160 that converts the first coordinatescalculated by the coordinate calculation part 159 into the secondcoordinates as the coordinates in the image data, and the output unit101 that outputs the second coordinates obtained by the coordinateconversion unit 160, and outputs the coordinates of the pointed locationby the pointing tool 12 as the coordinates in the image data. Thus, inthe PC 13 using the output coordinates or the like, the relativeposition between the pointed location and the image data may bespecified without being affected by the display mode of displayresolution, the largeness of the display area, or the like. In theprocess of obtaining the coordinates of the pointed location in theimage data, it is not necessary to directly associate the image dataitself with the pointed location, and it is not necessary to performcalibration even when the size of the image data or the like is changed.Therefore, the execution frequency of the calibration may be reduced.Thereby, the convenience of the projector 11 may be improved. Further,it is not necessary to execute the program for calibration at the PC 13side, and the burden on the user who is not familiar to the operation ofthe PC 13 may be lightened.

Further, the coordinate conversion unit 160 converts the firstcoordinates calculated by the coordinate calculation part 159 into thesecond coordinates based on the image location information as theinformation indicating the location of the display image with respect tothe displayable area, and thus, even when the image location informationas the information indicating the location of the display image withrespect to the displayable area changes, the coordinates of the pointedlocation by the pointing tool 12 may be correctly converted and output.

Furthermore, the coordinate conversion unit 160 converts the firstcoordinates calculated by the coordinate calculation part 159 into thesecond coordinates based on the resolution of the image data. Forexample, the coordinate conversion unit 160 performs coordinateconversion using the image location information reflecting the displayresolution of the projection unit 30 and the resolution of the imagedata. Thereby, even when the resolution of the image data changes, thecoordinates of the pointed location may be correctly converted andoutput.

In addition, the location detection part 151 detects the location of thepointing tool 12 on the screen SC based on the taken image taken by theimaging part 153, thereby, detects the pointed location in the effectiveprojection area 11B, and may promptly detect the pointed location.

Further, when the coordinates of the pointed location calculated by thecoordinate calculation part 159 are out of the area in which the imagedata is displayed, in other words, when the pointed location is out ofthe display image, the coordinate conversion unit 160 uses thecoordinates of the location near the pointed location within the area inwhich the display image is displayed as the converted coordinates, andthus, even when the location in the area with no image is pointed, mayoutput the coordinates of the pointed location. The output coordinatesare the coordinates of the location near the pointed location, and maybe processed by the PC 13 or the like in the same manner as that for thecoordinates of the pointed location.

Furthermore, when the coordinates of the pointed location calculated bythe coordinate calculation part 159 are out of the area in which theimage data is displayed, in other words, when the pointed location isout of the display image, the coordinate conversion unit 160 may notoutput the converted coordinates. In this case, the PC 13 may performthe operation corresponding only to the pointing in the locationoverlapping with the image.

In addition, in the case where the display mode of the image in thescreen SC is changed by execution of the processing on the image data bythe display control part 107 or the like, the coordinate conversion unit160 performs processing of converting the coordinates based on the imagelocation information that has been changed in response to the displaymode. The newly converted coordinates here are output by the output unit101. Thereby, proper coordinates may be constantly output with thechange in display mode of the display image.

Note that the above described embodiment is just an example of thespecific embodiment to which the invention is applied, but does notlimit the invention, and the invention may be applied as an embodimentdifferent from the above described embodiment. For example, in theembodiment, the configuration in which the converted coordinates areoutput to the PC 13 and the PC 13 draws the pointer 12A, the menu bar12B, etc. has been explained as an example, however, the invention isnot limited to that. An image processing unit 120 that generates imagesto be drawn and superimposed on the image data such as the pointer 12A,the menu bar 12B, etc. may be provided within the projector 11.

A projector 51 shown in FIG. 10 has the same respective functional partsas those of the projector 11 in the embodiment, and includes the imageprocessing unit 120 that draws the pointer 12A, the menu bar 12B, etc.in response to the pointed location of the pointing tool 12. The imageprocessing unit 120 includes an image processing part 122 that generatesan image superimposed on image data according to the coordinates inputfrom the coordinate conversion unit 160, and a frame memory 124 thatdevelops data when the image processing part 122 generates the image.

When the coordinate conversion unit 160 outputs the converted coordinatedata to the image processing unit 120, the image processing unit 120draws images of the pointer 12A and the menu bar 12B using the imageprocessing part 122, generates an image with the same resolution as thatof the image developed by the display control part 107, and outputs theimage to the image processing part 113. Here, the image output by theimage processing part 122 includes the image of the pointer 12A, themenu bar 12B, or the like. The image processing part 113 combines theimage input from the image processing part 122 with the image developedin the frame memory 115. Thereby, the image processing unit 120 maypromptly display the pointer 12A or the menu bar 12B superimposed on theinput image.

Further, in the configurations of the embodiments, the example in whichthe coordinate conversion unit 160 does not output the convertedcoordinates when the coordinates of the pointed location calculated bythe coordinate calculation part 159 are out of the area in which theimage data is displayed has been explained as an example, the inventionis not limited to that. For example, when the projector 11 determinesthe type of the externally input signal, when the projector 11temporarily stops the projected image, when the projector 11 interruptsimage projection, or the like, the coordinate conversion unit 160 maynot output the converted coordinates. Note that the projector 11 mayinterrupt image projection under the control of the control unit 103when the projection system 33 is shielded by a movable shielding part(not shown) such as a shutter provided in front of the projector 11,when a command for interrupting image projection via the operation partsuch as the operation panel 41 or the remote is received, or the like.

Furthermore, in the configurations of the embodiments, the imaging part153 and the image control part 155 of the location detection unit 150may be replaced by a digital camera externally connected to theprojector 11. The digital camera in this case may execute imaging underthe control of the control unit 130 and output taken image data to thelocation detection processing part 157. A general-purpose interface suchas a USB may be used as the interface connecting the digital camera andthe projector 11, and the digital camera may be easily realized.

In addition, in the configurations of the embodiments, the pointing tool12 is not limited to one having the rod shape or the pen shape, but, forexample, a finger of the user may be used as the pointing tool 12 andits pointed location may be detected. Any of the finger of the user anda device other than the finger of the user may be detected as thepointing tool 12.

Further, in the configurations of the embodiments, the configuration inwhich the location detection unit 150 detects the pointed location bythe pointing tool 12 based on the taken image data has been explained asan example, however, the invention is not limited to that. For example,a pressure-sensitive or capacitance touch panel may be provided on thescreen SC as the display surface or a display screen in other displaysystems, and the touch panel may detect contact of the user's finger, arod-like member, or the like as the pointing tool 12.

Furthermore, in the embodiments, the configuration in which the PC 13and the projector 11 are wired-connected by a cable or the like has beenexplained as an example, however, the connection form between theprojector 11 and the PC 13 is arbitrary. For example, the projector 11and the PC 13 may be connected to each other via wireless communicationusing a wireless LAN or the like or wired communication using ageneral-purpose data communication cable such as a USB, a wired LAN, orthe like, and may transmit and receive image data and coordinate data.

In addition, in the embodiments, the configuration in which the lightmodulator 32 uses the three transmissive or reflective liquid crystaldisplay panels corresponding to the respective colors of RGB as unitthat modulates the light generated by the light source has beenexplained, however, the invention is not limited to that. For example, asystem combining one liquid crystal display panel and a color wheel, asystem using three digital mirror devices (DMDs), a DMD system combiningone digital mirror device and a color wheel, or the like may beemployed. Here, in the case where only one liquid crystal display panelor DMD is used as the display unit, the member corresponding to thecombining system such as the cross dichroic prism is unnecessary. Otherdevices than the liquid crystal display panel or the DMD may be employedwithout difficulty as long as they may modulate the light generated bythe light source.

Further, the display device of the invention is not limited to theprojector that projects images on the screen SC. The image displaydevice of the embodiment of the invention includes various displaydevices such as self-emitting display devices of a liquid crystalmonitor or a liquid crystal television that displays images on a liquidcrystal display panel, a monitor device or a television receiver thatdisplays images on a PDP (plasma display panel), or a monitor device ora television receiver that displays images on an organic EL panel calledOLED (Organic Light-emitting diode), OEL (Organic Electro-Luminescence),or the like. In this case, the liquid crystal display panel, the plasmadisplay panel, the organic EL display panel correspond to a displayunit, and its display screen corresponds to the display surface. Morespecifically, the entire area in which images can be displayedcorresponds to the effective projection area 11B and the projectablearea 11A of the embodiments.

Furthermore, the respective functional parts of the projectors 11, 51shown in FIGS. 2 and 10 and the respective functional parts of the PC 13shown in FIG. 3 show functional configurations realized by cooperationof hardware and software, and the specific mounting form is notparticularly limited. Therefore, it may be not necessary that hardwareindividually dealing with the respective functional parts is mounted,and obviously, one processor may execute programs and realize thefunctions of the plural functional parts. Or, part of the functionsrealized by software in the embodiments may be realized by hardware orpart of the functions realized by hardware in the embodiments may berealized by software. In addition, specific detailed configurations ofthe other respective parts of the display system 10 including theprojector 11 and the PC 13 may be arbitrarily changed without departingfrom the scope of the invention.

Further, the control program 105A that has been stored in the memoryunit 105 in the embodiments may be downloaded from another deviceconnected to the projector 11 via a communication network, or thecontrol program 105A may be recorded in a portable recording medium andthe respective programs may be read out from the recording medium andexecuted. Similarly, regarding the display control program 13A stored inthe PC 13, the PC 13 may download the display control program 13A fromanother device and execute it or the PC 13 may read out the displaycontrol program 13A recorded in a portable recording medium and executeit.

Furthermore, in the embodiments, the configurations in which onepointing tool 12 is used has been explained as an example, however, thenumber of pointing tools is not limited in the invention. That is, theinvention may use two or more pointing tools at the same time. In thisregard, plural pointing tools 12 may be detected by one locationdetection unit 150, or all pointing tools 12 may be detected by locationdetection units 150 in the same number as the number of pointing toolsor in the larger number than the number of pointing tools, the locationdetection unit 150 that can detect the user's finger as the pointingtool 12, or the location detection unit 150 that can detect anotherdevice than the user's finger as the pointing tool 12 may be provided.Not all of the location detection units 150 may be provided in theprojector 11. For example, the projector 11 may include one locationdetection unit 150 and at least one location detection unit 150 may beprovided outside of the projector 11.

In addition, in the embodiments, the configurations in which thelocation detection unit 150 detects the coordinates pointed by thepointing tool 12 have been explained, however, the information detectedby the location detection unit 150 is not limited to the coordinatespointed by the pointing tool 12. The location detection unit 150 maydetect other information which pointing devices (mouse, digitizer, andso on) can detect, and output the information to the PC 13. For example,same as devices included in USB HID (human interface device) class(mouse, digitizer, and so on), the location detection unit 150 mayoutput the coordinate information and the other information(forinstance, information which indicates whether the operation portion ofthe devices is operated or not). The projector 11 may output thecoordinate information and the other information to the PC 13 via USBcommunication, a LAN, or the like, and the output method may be wiredcommunication or wireless communication.

Further, in the embodiments, the configurations in which the projector11 includes the location detection unit 150 have been explained as anexample, however, all or part of the configuration corresponding to thelocation detection unit 150 may be realized by another device than theprojector 11. For example, the projector according to the invention maybe adapted to connect to a digital camera having functions correspondingto the imaging part 153 and the image control part 155 and may acquiretaken image data from the digital camera. Furthermore, the locationdetection unit 150 may be another device than the projector 11 or the PC13. In this case, the location detection unit 150 may be a deviceindependent from the projector 11. In addition, the location detectionunit 150 may further has a function corresponding to the coordinateconversion unit 160.

What is claimed is:
 1. A display device comprising: an image input unitthat receives image data from an image supply device; a display unitthat displays a display image on a display surface based on the imagedata; a location detection unit that detects a pointed location withrespect to the display image on the display surface; a coordinatecalculation unit that calculates first coordinates as coordinates of thepointed location in a displayable area on the display surface; a drawingunit that draws an image based on the first coordinates calculated bythe coordinate calculation unit; and an output unit that outputscoordinates to the image supply device based on the first coordinates.2. The display device according to claim 1, further comprising: acoordinate conversion unit that converts the first coordinatescalculated by the coordinate calculation unit into second coordinates ascoordinates in the image data, wherein the output unit outputs thesecond coordinates obtained by the coordinate conversion unit.
 3. Thedisplay device according to claim 2, wherein the coordinate conversionunit converts the first coordinates calculated by the coordinatecalculation unit into the second coordinates based on image locationinformation as information indicating a location of the display imagewith respect to the displayable area.
 4. The display device according toclaim 3, wherein the coordinate conversion unit converts the firstcoordinates calculated by the coordinate calculation unit into thesecond coordinates based on resolution of the image data.
 5. The displaydevice according to claim 2, wherein the location detection unit detectsa location of a pointing tool on the display surface based on a takenimage obtained by taking the display surface using an imaging unit, andthereby, detects the pointed location in the displayable area.
 6. Thedisplay device according to claim 2, wherein the coordinate conversionunit uses coordinates of a location near the pointed location in theimage data as converted coordinates when the pointed location is notcontained in the display image.
 7. The display device according to claim2, wherein the coordinate conversion unit does not output convertedcoordinates when the pointed location is not contained in the displayimage.
 8. The display device according to claim 2, wherein, when imageprocessing is executed on the image data and a display mode of thedisplay image on the display surface is changed, the coordinateconversion unit converts the first coordinates into the secondcoordinates based on the image location information changed in responseto the display mode.
 9. The display device according to claim 2, furthercomprising: a display control unit that detects a location of thedisplay image with respect to the displayable area.
 10. The displaydevice according to claim 1, further comprising: a calibration executionunit that executes a calibration and obtains a correspondencerelationship between coordinates in a taken image and coordinates in thedisplayable area on the display surface, the taken image being obtainedby taking the display surface using an imaging unit.
 11. The displaydevice according to claim 1, wherein the output unit outputs coordinatesto the drawing unit based on the first coordinates.
 12. The displaydevice according to claim 2, wherein the coordinate conversion unitconverts the first coordinates calculated by the coordinate calculationunit into the second coordinates based on image location information asinformation indicating a location and a size of the display image withrespect to the displayable area, and when image processing is executedon the image data, and the size of the display image with respect to thedisplayable area is changed, the coordinate conversion unit converts thefirst coordinates into the second coordinates.